WORIJ3 INTELL£CTUAL PROPERTY ORGANIZATION 
Intcmational Bureau 




PCX 

INTERNATIONAL APPLICATION PUBUSHED UNDER THE PATENT CX)OPERATION TT^TY (PCT) 



(51) International Patent CIa5sificati(Mi ^ : 


Al 


(11) International Poblicatian Number: 


WO 94/16525 


Ha4N 7/08 


(43) International Pablkation Date: 


21 My 1994 (21.07.W) 



(21) Internatioaal Application Number: PCIAJS94/00206 

(22) [ntematioaal F^ing Date: 6 Janiiary 1994 (06i)l.94) 



(30) Priority Data: 
CW003,48I 



12 January 1993 (12.01.93) 



US 



(71) Applicant: VISUAL AUTOMATION SYSTEMS, INC. 

[US/US]; 801 South Grand Avaiuc. Los Angeles, CJl 
90017 (US). 

(72) Inventor; LEGATE, Kim, R.; 25S50 Turqiiesa, Valencia, CA 

91355 (US). 

(74) Agents: KONRAD* William, K. et al.; Spensley Horn Jubas & 
Lubitz, Fifth floor, 1880 Century Park East, Los Angelea, 
CA 90067 (US). 



(81) Designated States: AT, AU, BB, BG. BR, BY, CA, CH, CN, 
CZ, DE, DK, ES, H, GB, HU, JP. KP, KR, KZ, LK, LU, 
LV, MO, MN, MW. NL, NO, NZ, PL, FT, RO, RU. SD, 
SE, SK, UA UZ, VN. European patent (AT, BE. CH, DE, 
DK, ES, FR, GB, GR, IE, IT. LU, MC, NL, FT, SE), OAH 
patent (BF. BJ, CIF, CG. CI, CM, GA, GN, ML, MR, NE, 
SN. TO. TG). 



Published 

With intematicfftal search report. 

Before the expiration of the time limit for amending the 
claims and to he reptiblished in the event cf the receipt of 
amendments. 



(54) Tttle: VIDEO FACILITY MANAGEMENT SYSTEM 



4D*c— His 



y. , 







JP] — m 



SAFE TITLE BORDER i 

SAFE ACTItN BORDER 406' 



(57) Abstract 

Apparatus and method for tracking the Sow of tele vision programs within a program distribution system, each program including 
video signal whicb contains information to generate a television picture* the distribution system including a plurality of units through which 
the telcvisioa programs flow, by operating a first signal processing unit connected within the system for adding signal elements to the video 
signal in order to add pattems of symbols (404) to the television pictme (402) such that die symbols (404) are disposed at predetemiined 
locations of the tekvisioD picture (402) and a plurality of successive pactBms constituting a code uniquely identifying the program; and 
a second signal proces^g unit connected within the system for delecting signal elements previously added to a video signal and for 
identification of the program which included the video signaL 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the firont pages of pamphlets publishing international 
applications under the PCT. 



AT 


Austria 


GB 


UoUed Knigdam 


MR 


Maurtuma 


AU 


AufitraliA 


GE 




MW 


MaUwi 


BB 




GN 






Kiger 


BE 


Belgban 


GR 


Greece 


NL 


NcA^Flaocta 


BF 


Bortina Fuo 


HU 


Hongsy 


NO 


Norway 


BG 




IE 


Ire Land 


NZ 


NcwZraUod 


BJ 


Bcoia 


rr 


Ualy 


PL 


pDlAOd 


BR 


Brazil 


JP 


JapiD 


FT 


Foxtugil 


BY 


BeUra 


KE 


Ksgya 


RO 




CA 


Cuuda 


KG 


Kyigyuan 


RV 


RiosLn Federation 


CF 


Ctotrtl AfriCAd Republic 


KP 


Democndc Poople^s RepobUc 


SD 




CG 


Congo 




of Katca 


SE 


Sweden 


CH 


SwilirrlaDd 


KR 


Reptri>bc of ]&>rea 


SI 


Slovenia 


cr 


Cfice d'lvoire 


KZ 


KazaUistaQ 


SK 


Slovakia 


CM 


CuDcraoD 


U 


LtccfatcosteiD 


SN 


Scoei^ 


CN 


Chioa 


LK 


SiiLBBka 


TD 


Qiad 


cs 


Cziediotlovaki& 


LU 


Lujlcmboiirg 


TG 


Tbgo 


cz 




LV 


Latvia 


TJ 


TijikkataD 


DE 


Gci'iuaoy 




bdooaco 


TT 


Thiiklad and Tobago 


DK 


DctiiiuuIl 


MD 


RfpuMlc of MoUova 


OA 


Uknooe 


ES 


Spain 




Madagtecur 


VS 


UcitEd Sasea of America 


n 




ML 


M«U 


UZ 


UzbekiBt&it 


FR 




MN 


MoDgo&a 


VN 


VtctNam 


GA 


Gmboo 











wo 94/16525 PCT/US94/00206 



1. 

VIDEO FACILITY HANA6EHENT SYSTEM 



BACKGROUND OF THE TmTENTION 

TUe present invention relates to the management . of 
television facilities, particularly for the purpose of 
tracking television programs stored, in recorded form, and 
5 being transmitted within, the facility, as well as 
programs being received and transmitted by the facility. 

Television programs, prior to broadcast or 
transmission over a cable system, may be conducted over 
a wide variety of paths within a facility and between 

10 facilities- Up to the present, no system or procedures 
have been available to reliably monitor the flow of such 
programs in real time. As a consequence, those managing 
such facilities generally must rely on the assumption that 
all operating personnel have taken the necessary steps to 

15 route the programs in the desired manner. At present, one 
of the most feasible procedures for determining what has 
actually been broadcast or distributed by a cable system 
is to make an off air recording of all programs which have 
been broadcast or distributed to consumers. In other 

20 words, this procedure only allows the required 
determination to be made offline and hence not in real 
time. 

The television industry has implemented several 
different computer systems and associated software 

25 designed to achieve specific objectives, such as the 
preparation of a log of programs to be broadcast or 
distributed. Some accounting programs have been developed 
which make use of manually entered data derived from the 
log of the programs intended to be broadcast as well as 

30 discrepancy reports from operators on duty at the time of 
broadcast and the viewing of off air recordings. In 
addition, some limited automation of on air programs has 
been attempted. However, to date, the capability of such 
systems is limited because, generally, they are not 

3 5 capable of automatically identifying the programs which 
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are being routed through, or are stored in, a facility. 
Thus, the timely distribution of television programs to 
consumers, either by broadcasting or by delivery to a 
cable network, is for the most part, if not entirely, 
5 dependent on, and in most instances can only be verified 
by, the responsible operating personnel. 
SUI^MTURY OF THE INVENTION 

It is a primary object of the present invention to 
automatically monitor the routing of television programs, 

10 or their elements, within a facility and between 
facilities in order to permit the automatic generation of 
a record of the path followed by each television program 
or element and the time at which each program or element 
reached each distribution point along its routing path. 

15 A more specific object of the invention is to provide 

the video signals of each program with information which 
identifies the program and which can be decoded in a 
manner to be supplied to a computer system, thereby 
allowing real time management of a television facility, 

2 0 Another specific object of the invention is to 

provide such program identifying information in the region 
of a video signal which contains picture information, but 
which does not interfere with viewing of the associated 
program on a home receiver. 

25 The above and other objects are achieved according to 

the invention by the provision of apparatus for tracking 
the flow of television programs within a program 
distribution system, each program including a video signal 
which contains information to generate a television 

30 picture, the distribution system including a plurality of 
units through which the television programs flow, the 
apparatus comprising: first signal processing means 
connected within the system for adding signal elements to ^ 
the video signal of a program in order to add patterns of * 

3 5 symbols, referred to hereafter as icons, to the television 

picture such that the symbols are disposed at 
predetermined locations of the television picture and a 
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plurality of successive patterns constitute a code 
uniquely identifying the program; and second signal 
processing means connected within the system for detecting 
signal elements previously added to a video signal and for 
5 producing, in response to such detection, an 
identification of the program which included the video 
signal. 

Objects according to the invention are further 
achieved by a method for tracking the flow of television 

10 programs within a program distribution system, each 
program including a video signal which contains 
information to generate a television picture, the 
distribution system including a plurality of units through 
which the television programs flow, the method comprising: 

15 adding, in first signal processing means connected within 
the system, signal elements to the video signal of a 
program in order to add patterns of symbols, referred to 
hereafter as icons, to the television picture such that 
the symbols are disposed at predetermined locations of the 

20 television picture and a plurality of successive patterns 
constitute a code uniquely identifying the program; and 
detecting, in second signal processing means connected 
within the system^ signal elements previously added to a 
video signal and producing, in response to such detection, 

25 an identification of the program which included the video 
signal , 

Because the elements of a television program are 
routed through a number of pieces of signal processing 
equipment before broadcast or delivery to a consumer cable 

30 system, these elements are subject to various artifacts 
and distortions not found in the original program 
material. Additional artifacts and distortions will, of 
course, be introduced into television signals during 
broadcast or distribution over a consumer cable system. 

3 5 All portions of a video signal not contained within the 
active picture window, also known as the scattered image 
area, which contains picture information are subject to 



^1 
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and indeed are at some point reshaped, retimed, and/or 
replaced, which can make any information scheme using 
these areas often unreliable at best. The most reliable 
place to insert any form of identification scheme is 
5 therefore in the active picture area. 

One major problem associa'ted with using the active 
picture area is that the information may be inserted at 
a location which is objectional to the viewer. 

The present invention obviates, or at least 

10 substantially ameliorates, the first of these problems by 
utilizing icons which are relatively small compared to the 
picture as a whole, and by inserting these icons into 
video signals so that, in a picture produced by those 
signals, these icons will be located in an area of the 

15 picture which will not be seen on a properly adjusted home 
television receiver. This area will be referred to 
hereafter as the icon area. The outer boundary of the 
icon area coincides with the boundary of the above- 
mentioned scattered image area and the inner boundary of 

20 the icon area preferably coincides with the boundary of 
an area known in the art as the safe title area, and more 
preferably with the boundary of an area known in the art 
as the safe action area, which is slightly larger than the 
safe title area. The form and dimensions of these areas 

25 will be described below. In a properly adjusted home 
television receiver, the picture width amd height 
adjustments are set to cause electron beam scanning to 
correspond to an area slightly larger than the picture 
tube usable area. Ideally, the adjustments are set so 

30 that safe action area of the picture corresponds to the 
picture tube usable area. 

A second problem is that distortions and artifacts 
exist when the program is viewed by the consumer. While 
these distortions may not be objectionable to the 

35 consumer, they may still cause unreliability in digital 
systems. 

For example, because of the inherent characteristics 
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of equipment which is currently employed for processing 
video signals, there are certain obstacles to accurate 
placement of information at defined locations of a picture 
area. As regards accurate positioning in the vertical 
5 direction, picture line counting is performed using the 
vertical sync as a reference, and digital video processors 
and video tape recording time base correctors replace both 
vertical and horizontal sync pulses. Therefore, the 
accuracy with which a particular picture line can be 

10 determined is a function of each individual piece of 
equipment employed during the processing. A common 
problem with some of the more popular video tape recorders 
is that they can lock to a video signal one line late. 
The time base corrector will then add sync information so 

15 as to shift the picture one line vertically. These shifts 
go unnoticed by the viewer because the missing line is 
outside of the picture area normally viewed on a home 
receiver. 

With regard to picture element positions in the 

20 horizontal direction, the position of any picture element 
along a line of actual video is referenced to the leading 
edge of the horizontal sync pulse for that line. The 
types of video processing equipment referred to above and 
even the older analog processing amplifiers have the 

25 ability to change this relationship. Again, the viewer 
is not affected by such changes because they have no 
reference as to the actual location of the picture edge. 
In practice, this error is held to approximately one-half 
cycle of the subcarrier because larger amounts of picture 

30 phase error would cause horizontal blanking to expand 
beyond permissible limits. 

Existing equipment also presents problems relating to 
analog brightness level and linearity of signal 
amplification. Even distribution amplifiers, which are 

3 5 the simplest element employed in television processing 
systems, can introduce significant analog errors into 
signals which they process. These errors include errors 
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relating to differential gain, signal compression, high 
or low brightness levels, nonlinear transfer functions, 
etc- More complicated equipment can introduce even higher 
error levels. Therefore, when digitizing a signal which 
5 has passed through any of these devices, quantizing errors 
are liXely to occur. 

The elements, or icons, employed to provide 
information in accordance with the present invention have 
the capability of retaining their information content even 

10 when the video signals in which they are contained are 
subjected to the various shifts and distortions mentioned 
above. The icons are constituted by simple patterns which 
have sufficient redundancy to assure reliable detection 
even after having been subjected to vertical and 

15 horizontal displacements of the type described above. In 
order to assure reliable decoding even if the icons have 
been subjected to analog level quantizing enrors^ each 
picture frame includes, in a defined area, a specific 
reference icon having regions which represent picture 

20 black, white and gray levels. The difference between the 
known values of these levels and the pixel values 
encountered when reading the reference icon gives an 
indication of the magnitude of gain and linearity errors 
which occurred subsequent to encoding. This information 

25 can be used as offsets in reading the icons recovered from 
the other three icon areas of a picture frame. 

More specifically, the invention enables the problems 
relating to picture element position shifts and analog 
signal distortion to be overcome by using icon patterns 

3 0 which can be reliably detected and identified with the aid 
of currently available technology, employing fuzzy logic, 
even in the presence of high levels of noise or 
distortion. As opposed to detection techniques employing 
the classic Boolean logic, where each signal element can 

35 have only one of two values, O or 1, fuzzy logic can 
associate each signal element with a values in a 
substantially continuous range between o and l^ Therefore, 
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fuzzy logic pattern recovery does not rely on a perfect 
match. It returns a value as to the closeness of a match - 
Reasonable amounts of noise or distortion do not haJDfiper 
symbol recovery when the icons are constituted by patterns 
5 within parameters contemplated by the present invention, 
giving the system a robustness not found in Boolean based 
information systems . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing a conventional 
10 television facility equipped with components for 
implementing the present invention. 

Fig. 2 is a block diagram showing one of the 
components according to the invention provided in the 
system of Fig* 1* 
15 Fig* 3 is a block diagram of one unit of the component 

of Fig 2. 

Fig- 3.1 is a block diagram showing those portions of 
the unit of Fig- 3 for inserting icons into a video 
signal ; 

2 0 Fig* 3.2 is a block diagram showing those portions of 

the unit of Fig. 3 for reading icons from a video signals- 
Fig. 3-3 is a block diagram showing those portions of 
the unit of Fig. 3 for identifying icons read from a video 
signal; 

2 5 Fig- 4 is a pictorial view of a television picture 

screen , showing the location of identifying elements in 
accordance with the present invention - 

Figs. 5 •1-5.2 are pictorial views of portions of a 
variety of identifying patterns which can be employed in 

3 0 the practice of the present invention. 

Figs. 6-1-6-3 are timing diagrams illustrating the 
operation of the device of Fig. 3, 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 is a block diagram of a typical television 
3 5 distribution system which can be constructed in accordance 
with the present invention. The configuration of the 
exemplary system has been selected arbitrarily, simply for 
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purposes of illustration. It is to be understood that any 
television program distribution system, regardless of its 
complexity and/ or geographic extent, could serve as a 
basis for implementation of the invention. Furthermore, 
5 although the illustrated embodiment is described in 
connection with the N.T.S.C. standard, it should be 
appreciated that the present invention is applicable to 
all video formats, both those in present use as well as 
those formats which will be developed in the future. 

10 The system illustrated in Fig. 1 includes television 

signal processing components, such as a satellite receiver 
102 , a video tape recorder 104 and a television signal 
processing and control area, known in the industry as a 
break studio, 106. Satellite receiver 102 is connected 

15 to a receiving antenna 112 for receiving programs which 
are downloaded from a satellite and programs arriving at 
receiver 102 are conducted via a router 114 to other 
components of the system, such as the components 104 and 
106. Thus, a program arriving via a downlink at receiver 

20 102 can be recorded by recorder 104 or immediately sent 
out over the air via control unit 106. Alternatively, a 
tape carrying a program which has previously been recorded 
can be conducted from VTR 104 to control unit 106 via 
router 114 . 

25 The system as thus far described represents an 

operational, albeit simple, television station* In 
accordance with the present invention, specially designed 
nodes 12 0 are connected at each junction where television 
program signals enter, leave, or can be routed within, the 

3 0 station. As will be explained in greater detail below, 
each of these nodes 120 can be controlled to identify each 
television program, or transmission, which passes 
therethrough, and/ or to insert identifying indicia into 
each such transmission. 

35 The system shown in Fig. 1 is completed by units which 

are provided to utilize information provided in a video 
signal in accordance with the present invention, and which 
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are coupled to nodes 120 by a cable 122. These additional 
units include a supervisory computer system composed of, 
for example, a unit 13 0 which maintains a schedule of the 
daily work to be performed at the station^ a unit 132 
5 which provides a log of programming which will be 
broadcast by the station , a unit 134 performing accounting 
functions, and one or more unit^s 13 6 performing 
laiscellaneous tasks requiring information provided in 
accordance with the present invention. The supervisory 

10 computer system may also store information identifying the 
desired broadcast time of a program and use that 
information in conjunction with information indicating the 
location; i.e. the tape recorder, where the program is 
stored to issue control signals which begin playback and 

15 route the recorded output to the station transmitter. 

The station further includes an engineering network 
composed of a unit 14 0 which stores the identification of 
each program handled by the station^ a unit 14 2 which 
stores data relating to the technical quality of the 

20 picture and sound content of programming received and 
sent, a unit 144 which may perform miscellaneous tasks, 
a unit 14 6 which constitutes a file server that maintains 
an updated record of relevant data relating to the 
programming received and broadcast by the station and a 

25 unit 148 which permits exchange of information between 
units 130-136, on the one hand, and units 140-146, on the 
other hand. 

The term "program" appearing in various legends in 
Fig. 1, and elsewhere herein, is employed to mean an 

30 identifiable television program element, i.e, a completed 
TV show or production, including commercials or public 
service announcements, which have been assembled for 
broadcast over the air or via a cable system. Of course, 
such a program may also be a live broadcast, such as of 

35 a news story or a sporting event. The structure and 
arrangement of conventional broadcast facilities are 
described in the current edition of the NAB (National 
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Association of Broadcasters) Engineering Handbook, 

Fig. 2 is a block diagram showing one embodiment of 
a node 12 0 according to the present invention. The node 
includes an interface 202 which connects the node to cable 
5 122. Interface 202 may be a local area network controller 
such as an interface marketed by Standard Microsystems 
Corp. under the model designation COM20020. interface 202 
is connected via a control bus 204 and an address /data bus 
206 to a CPU 210, a system memory 212 and a real time 
10 clock 214- CPU 210 is connected via a master/slave 
control bus 220 to one or more modules 222, 224, 226 and 
228. 

Module 222 is constructed and controlled for inserting 
identifying indicia such as "icons" into a video signal 

15 or to read icons which have previously been inserted into 
the video signal. Module 222 includes an analog portion 
and a digital portion which can derive digital signals 
representative of icons in a video signal processed in the 
analog portion. The time at which a set of icons passes 

20 a particular node can be read from clock 214 and stored 
with data identifying the program and the node which it 
passed. 

Module 22 4 is a machine control module which can be 
configured to control the operation of a component of the 

25 television distribution system in response to information 
derived from the video signal by module 222- Examples of 
components which are adapted for machine control include 
character generators, routers and video tape recorders 
such as the Ampex VPR^2B and the Ampex VPR-250 and other 

3 0 addressable components. 

Module 22 6 is a digital audio icon interface which can 
modify the audio portion of a television program in 
response to information which has been added to the video 
signal thereof and decoded in module 222, Module 228 is 

3 5 a station time reader which receives an input representing 
the current time and associates a time indication, or 
stamp, with each set of data obtained from module 222. 
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Machine contirol 2 24 and interface 22 6 are optional 
components which can be provided when desired. Certain 
of the capabilities which will be created by provision of 
such components will be described below, after the device 
5 and procedure for inserting information into a video 
signal and reading such information have been described. 
Fig. 3 is a schematic block diagram of the icon 
encode/decode module 222. Figs. 3.1-3.3 depict various 
portions of the module 22 2 in accordance with the function 
10 performed. Figs, 6.1-6.3 are timing diagrams showing the 
waveforms of signals at certain points in the circuits of 
Figs. 3.1-'3.3, respectively. The locations in the 
circuits of Figs. 3.1-3.3 where the signals of Figs. 6.1- 
6 . 3 appear are indicated by letters corresponding to those 
15 identifying the corresponding waveforms of Figs, 6-1-6,3, 
Turning first to Fig. 3.1, there is illustrated one 
preferred embodiment of the portion of module 222 which 
interacts with a video signal in order to insert 
information into the signal. The circuit shown in Fig. 
20 3.1 includes a video buffer 302 and a key amplifier 304 
which are connected together in series in the associated 
television signal path. The video signal applied to the 
input of amplifier 3 02, waveforms ^ and aj_ in Fig. 6.1, 
is conducted without modification to a first signal input 
25 of amplifier 304- Key Amplifier 304 is a conventional 
element used in television signal processing systems. It 
is basically a high speed video switch that passes , in 
this case, either the unmodified input video or video in 
which identifying information has been inserted after a 
30 D/A conversion process. In the illustrated embodiment, 
this identifying information is preferably in the form of 
graphic symbols or "icons" as explained below. Selection 
of the input signal to amplifier 304 is controlled by the 
signal "Insert", waveform s of Fig. 6.1., which is 
35 generated by a sync stripper/ clock generator circuit 306, 
The video signal leaving buffer amplifier 302 is 
conducted to the sync stripper/clock generator 306 which 
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has a basic tiraing element. The circuit 306 includes a 
type liMlSBl video sync separator chip marketed by National 
Semiconductor, the output of which (designated INSERT) is 
connected to a subsidiary clock generator 310 as well as 
5 the key amplifier 304. The amplifier 304 has a second 
video input which receives one or more icons which can be 
added to the video signal. The icon data is provided to 
the second video input of amplifier 3 04 by circuitry which 
includes a FIFO memory 32 0 and a digital /analog converter 

10 322. Signals representing the desired icon patterns are 
read from memory 320, are converted to analog form in 
converter 32 2 and are added to the video signal at the 
appropriate points in time* 

Devices 3 06 and 32 0 are controlled by a microprocessor 

15 330 which has an associated memory 332. Microprocessor 
3 30 is connected to clock generator 3 06 via a control bus 
340. Microprocessor 33 0 is connected to memory 3 32 and 
memory 320 via a data bus 342 and an address bus 344* 

A signal up_insert_enabl« (wavefoinn e in Fig. 6.1) is 

20 generated in device 3 06 and supplied to microprocessor 33 0 
via control bus 34 0, This signal alerts the 

microprocessor to load FIFO memory 32 0 with appropriate 
icon patterns stored in the microprocessor's associated 
memory 332, Microprocessor 330 does not have to load 

2 5 memory 320 with all icons to be stored into a frame at one 

time since the FIFO memory 320 can be read from and 
written to at the same time. In the illustrated 
embodiment, the data for only the first line of icons to 
be inserted into the frame is loaded into the FIFO memory 

3 0 at a time. 

At the desired location in the field or fields of the 
frame of the video signal, the clock generator 306 under 
the control of the microprocessor 330, activates the 
timing signal INSERT which causes the clock generator 310 
35 to issue a series of rapid clock pulses to the FIFO memory 
320 and the D/A converter 322. The FIFO memory 320, in 
response to these clock pulses, outputs the previously 
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Stored icon information, a pixel at a time, which is 
converted to analog form by the converter 322 and is then 
inserted into the video signal by the key amplifier 304* 
In this manner^ information identifying a program may 
5 be added to the video signal of the program. The 
information is in the form of signal components which will 
appear at selected locations in the resulting television 
picture as icons, or graphic symbols. These icons can 
appear in a variety of patterns; each pattern represents 

10 a specific information unit, or datum; and a succession 
of these patterns provides the desired information. 

In the illustrated embodiment, for each icon, a first 
set of pixels is inserted in successive lines of Field 1 
of the frame as shown for example in Fig. 5.1- To 

15 complete the icon, a second set of pixels is inserted in 
associated successive lines of Field 2 of the same frame . 

To read the icons, the video signals in the 
preselected areas reserved for icons are converted into 

20 digital pixel signals and are then stored in memory 314 
of Fig. 3.2 at locations such that all pixels of Fields 
1 and 2 for a given icon can be read out together in a 
selected order. Figure 4 shows an example of four such 
icon areas 404a-4 04d of a video frame • It is recognized 

25 that the icons may be placed in other areas of the frame. 

The vertical positions and extents of icon areas 4 04 
are determined by counting the horizontal sync pulses from 
the start of each field and producing vertical gate pulses 
in response to selected horizontal sync pulses. In the 

30 illustrated embodiment, two icons are inserted into lines 
22-27 and two are inserted into lines 257-262 of each 
frame for a total of four icons per frame, as shown in 
Fig. 4. Since each frame has two interleaved fields, each 
icon has a vertical height of 12 lines as shown in Fig. 

35 5.1. It should be appreciated of course that various 
nximbers of icons can be inserted into a particular frame. 
For example, three icons can be stacked in the same 
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tiorizontal position in each corner of the frame for a 
total of 12 icons per frame. 

The horizontal positions and extents of icon areas 4 04 
are determined by producing a train of pixel clock pulses 
5 which divide each scan line into a plurality of pixel 
locations, counting these pixel clock pulses and producing 
horizontal gate pulses in response to selected pixel clock 
pulses. The coincidence of vertical and horizontal gate 
pulses correspond to an icon area* 

10 Fig. 6.2 contains timing diagrams showing the 

waveforms of signals at certain points in the circuit of 
Fig. 3.2. In Fig 6*2, waveforms a and aj_ show two 
successive parts of one field of a color television signal 
which starts with a vertical blanking interval^ to the 

15 left of waveform a of Fig. 6.2 during which equalizing 
pulses and sync pulses are provided, followed by eleven 
picture lines (picture lines 10-20) which will not appear 
on the television screen. The vertical blanking interval 
is followed by an active video period composed of 242% 

20 picture lines. As is known, each picture line includes 
a horizontal sync pulse 610, a color burst 612 and a 
picture information region 614. The region 614 containing 
picture information is depicted as a pulse in picture line 
21 but is not depicted for the other picture lines. 

25 

Fig. 6.2 shows, at waveform g, the field 
identification pulse, odd^even, extracted from each field 
signal by device 3 06 of. Fig. 3.2. The odd_6ven signal is 
conducted to the reset input RST of a counter 309, the 

30 output of which is a ten bit parallel digital signal that 
represents the pixel address within the frame. Thus, the 
counter 3 09 resets and generates sequential ten bit pixel 
addresses when the field changes from odd to even and vice 
versa. It will be appreciated that the pixel frame 

3 5 addresses of each icon pixel will not be contiguous from 
line to line. Conseguently , the ten bit pixel address 
generated by the counter 3 09 is conducted to a PROM device 
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312 which re-maps the pixel frame address to contiguous 
iBemory addresses of the memory 314, As a result ^ the 
pixels of each read icon entering memory 314 is stored 
into contiguous memory locations. 
5 In Fig. 6.2 there are at waveform shown horizontal 

sync pulses, separated h^syne, extracted from the video 
signal of waveforms a and a/. This signal is generated 
within device 306 of Fig. 3.2. By counting the separated 
horizontal sync pulses of waveform h of Fig- 6.2, a signal 

10 line_gate as shown in waveform c of Fig» 6*2 , is produced 
which provides a first gate pulse synchronized with 
picture lines 22-27 at the top of the picture frame and 
a second gate pulse synchronized with picture lines 2 57- 
262 at the bottom of the picture frame. This signal is 

15 also produced in device 3 06 of Fig. 3,2, 

During each gate pulse line_gate, the generator 306 
produces a series of pulses, identified as pixel_gate 
pulses, which are shown in waveform d of Fig. 6,2- Each 
pixel_gate pulse coincides with one horizontal row of a 

20 respective icon area. As previously mentioned, the frames 
of the illustrated embodiment have four icon areas, one 
icon area at each corner of the frame. Because each 
picture line passing through an icon area passes through 
two icon areas in the illustrated embodiment, alternate 

25 ones of these pixel_gate pulses coincide with a defined 
part of the beginning and end, respectively of each 
picture line. Each of these gate pulses is used to 

gate two series of gated__pixel_c locks generated by the 
generator 306, which are shown in waveforms e and f of 

30 Figs. 6.2. The gated_pixel_cloc)cs e (phase 1) clock an 
A/D converter 308 so that only that portion of the video 
signal corresponding to a preselected icon area is 
converted into digital pixel data. The gated_pixel_clocks 
f (phase 2) clock the counter 309 which generates the 

3 5 pixel frame addresses described above which are mapped by 
the PROM 312 to contiguous memory addresses. Each pixel 
written into memory 314 is constituted by an eight-bit 
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word written into memory 314 in parallel. 

Referring now to Figs. 3.3 and 6.3, icons are 
identified by comparing icon pixel data stored in 
addressable digital memory 314 during a previous icon read 
5 operation with icon pixel data stored in a pattern RAM 
318- For this purpose, at the end of each picture frame, 
the pixel data read from the picture frame and stored in 
memory 314 are outputted to a shift register 315 which 
delivers the pixel data a bit at a time to a pattern 

10 recognition unit. In the illustrated embodiment, the 
pattern recognition unit is implemented with a fuzzy 
pattern comparator 316 and pattern RAM 318. One suitable 
embodiment of comparator 316 is a device marketed by 
American NeuraLogix, Inc. , of Sanford, FL, under model 

15 designation NLX 110, This device has eight pattern data 
inputs, so that RAM 318 may be provided with eight data 
outputs for simultaneously supplying up to eight selected 
patterns to comparator 316. 

The NLXllO integrated circuit allows for simultaneous 

2 0 comparison of up to eight known patterns with one unknown 

pattern. To make maximum use of this architecture, the 
illustrated embodiment of the present invention utilizes 
eight different icon patterns. One icon pattern is 
designated as a reference icon pattern and is located in 
25 the upper left hand corner of the active picture window, 
as indicated at 404a in Fig. 4. This reference icon 
pattern is used by control microprocessor 33 0 to set the 
error threshold of the NXXllO pattern comparator 316 for 
the next three icons, present in areas 4 04b, c and d, 

3 0 thereby tracking out noise or distortion on the incoming 

icons . 

The other seven selected icon patterns constitute an 
icon primitive set. Using the formula for the number of 
permutations of N (=7) patterns taken R (=3) at a time 
35 yields 210 different pattern permutations possible per 
picture frame. The first 128 permutations can be assigned 
to correspond to the seven bit ASCII code for plain text 
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transmission. The remaining values can be assigned to 
special purposes. With this scheme, when program 
identifying information is being provided^ three different 
icon patterns plus the reference icon pattern will be 
5 present in a frame. Thus, each picture frame can provide 
one alphanumeric identification ch.aracter. Given the 
conventional television frame rate of 30/second, icons 
providing complete identifying information need be present 
in a television program signal for only a few seconds. 

10 By extension, increasing the number of different icon 

patterns would increase the quantity of information 
contained in each frame. However, it is anticipated that 
the complexity of the circuitry and control programming 
would likely increase as the nximber of different icon 

15 patterns increases. In addition, an increase in the 
number of different icon patterns would make visual 
interpretation by station personnel more difficult. 

The types of icon patterns employed in the 
illustrated embodiment of the present invention have 

20 characteristics which enable them to be identified, or 
decoded, by a fuzzy logic comparator with a high degree 
of reliability even when the video signal contains noise 
or distortion or when an error in the timing of the video 
signal prevents one or two rows or columns of icon pixels 

25 from reaching comparator 316. In the illustrated 
embodiment, these icon pattern characteristics include the 
following: there are only eight different icon patterns 
to be recognized, while each pattern is composed of 14 4 
pixels; each pattern consists of pixels having only two 

3 0 or three different values spaced relatively far apajrt on 
the gray scale; each pattern has no more than four changes 
in gray level value in both the horizontal and vertical 
directions; and, with the exception of two corners of each 
triangle pattern, a given gray level value is present at 

3 5 at least two successive pixel locations in both the 
horizontal and vertical directions. Therefore, the eight 
different patterns can be made to represent relatively 
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simple geometric shapes which differ sufficiently from one 
another to permit reliable identification of each pattern 
with comparator 316 set to a relatively low recognition 
threshold. 

5 In the illustrated embodiment, the icon information 

stored for a particular frame is identified, or decoded, 
within a period equal to 12 picture lines with 18.2 MHz 
shift clocks. As shown in Fig* 6.3, waveform 3 is the 
signal odd_even produced in device 306 to identify the 

10 current field of each frame. The state of this signal is 
inverted by the leading edge of each v^syne pulse (Fig, 
6-1) . At each positive transition of the signal Ddd_eveii, 
corresponding to the vertical blanking interval at the 
start of each odd field, device 306 generates a pulse 

15 compar amenable having a period of 12 picture lines, as 
represented by waveform h. 

During each pulse compare_enabXe , device 3 06 generates 
a train of four spaced pulses read^winner , represented by 
waveform i, and four spaced pulses compare^gate, 

20 represented by waveform ±, Each pulse compare_gate 
immediately precedes an associated pulse read^wlnner . 
During the period of each pulse compare^gate, device 3 06 
generates a train of 14 4 pulses of gated^cloclLS , shown in 
generalized form as waveform k. 

25 As shown by waveforms 1 and jn, each pulse of 

gated_^cIocks is composed of two trains of eight clock 
pulses^ each at a pulse rate of 18.2 MHz. In addition, 
device 3 06 generates a pulse mem_read, shown by waveform 
in, for each pulse of gated_clocks . Counter 319, Fig. 3-3, 

30 is reset by the odd_even pulse, waveform 3, at the start 
of every frame. Mem_read is supplied to the input of 
counter 319 which provides a ten bit pixel read address 
for memory 314 via mux 311, Fig. 3. 3. 

One of the trains of eight clock pulses, identified 

35 as waveform 1 (phase 1) , is supplied to shift register 315 
to shift pixel bits serially into comparator 316 and the 
other train, identified as waveform m (phase 2) , is 
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supplied to comparator 316 to trigger a comparison 
operation. Each pulse of phase 1 shifts a single pixel 
bit and each pulse of phase 2 triggers a comparison with 
respect to the previously shifted bit. The bits of phase 
5 2 are shifted in phase by 180*" from the bits of phase 1, 

Each bit arriving from shift register 315 is compared 
with an associated bit of each of the eight patterns 
supplied by RAM 318- After all bits of one icon have been 

10 compared, a corresponding read^winner pulse is delivered 
to microprocessor 330, and in response to that pulse, 
microprocessor 330 reads the comparison result out of 
comparator 316. This comparison result is in the form of 
a signal identifying the reference pattern which most 

15 closely matches the pattern represented by the bits 
delivered from register 315. This signal is supplied to 
memory 332 along with identification of the associated 
icon location in the active picture window. 

The trailing edge of ccmpare_enal>le generates, in 

20 microprocessor 330, a done signal indicating that all four 
icons of a frame have been individually identified as to 
pattern and location. The information, i.e. the 
character, which they represent can be identified, for 
example, by means of a look-up table in memory 332. 

25 A node in accordance with one aspect of the invention 

is disposed at the input side of the transmitter to read 
the identifying information stored in the television 
signal so as to provide the central data processing system 
of the facility with a contemporaneous and reliable 

30 indication that broadcast of the identified program has 
begun. This information will be stored together with an 
indication of the time that transmission began. 

In addition, after reading, the stored information may 
be deleted from the television signal prior to 

35 transmission. Alternatively, however, the identifying 
information, i.e. the icons, may remain in the television 
signal at the time it is broadcast or transmitted for use 



f 





wo 94/16525 ^ 



PCT/US94/00206 



20. 



5 



10 



15 



20 



25 



30 



in order -to determine viewing patterns at individual 
television receivers, for example in order to derive 
program viewership. For this purpose, each television 
receiver which is to be monitored would be equipped with 
a decoder containing appropriate portions of the node of 
Fig . 2 connected to the output of the receiver tuner or 
forming part of an external tuner. Identification data 
contained in each program to which the receiver is tuned 
could then be stored together with a time indication or 
could be transmitted, possibly immediately, via a 
telephone line to a control data collection point. As a 
consequence, program viewing ratings can be determined on 
a virtually instantaneous basis. 

When the identification data according to the 
invention is used to determine viewing patterns, it may 
be desirable to transmit such data at fixed intervals of, 
for example, five minutes during each program, as well as 
at the start of each commercial, and after each 
commercial, in order to monitor patterns of channel 
switching in the middle of programs. Such a technique for 
monitoring viewing patterns would eliminate ratings errors 
that result from incorrect reporting by viewers of the 
programs which they watched. 

Fig. 4 shows an active picture window having an outer 
border, or boundary, 4 02 delimiting the area of a video 
image formed by all picture information contained in a 
video signal, i.e. all portions of the video signal other 
than the horizontal and vertical blanXing portions. 
Within the active picture window there is a safe action 
area enclosed by an outer border, or boundary, 406, and 
within the safe action area there is a safe title area 
enclosed by an outer border, or boundary, 408. 

According to Specifications published by the Society 
of Motion Picture and Television Engineers as RP27. 3-1989, 
the safe action area will contain that part of the picture 
which it is desired to present to the viewer, i.e. all 
significant action in a television program, and the safe 



wo M/ld525 PCT/US94/00206 



21. 

title area will contain all of the more important 
information, such as titles, to ensure that this 
information will be visible on the majority of home 
television receivers. 
5 In further accordance with those specifications, the 

following dimensions are associated with the picture area, 
in terms of percentage of the vertical height of the 
active picture window enclosed by border 4 02: 

Horizontal width of active picture window - 
10 133%; 

Vertical height of safe action area - 90%; 
Horizontal width of safe action area - 120%; 
Vertical spacing of bottom edge of safe action 
area from bottom edge of active picture window - 
15 5%; 

Horizontal spacing of right hand edge of safe 
action area from right hand edge of active 
picture window - 6.67%; 

Radius of each corner of safe action area - 
20 24%; 

Vertical height of safe title area - 8 0%; 
Horizontal width of safe title area - 106%; 
Vertical spacing of bottom edge of safe title 
area from bottom edge of active picture window - 
25 10%; 

Horizontal spacing of right hand edge of safe 
title area from right hand edge of active picture 
window - 13.33% 

Radius of each corner of safe title area - 
30 21%. 

Figure 4 further shows four small rectangular icon 
areas 404 located at the four corners of the active 
picture window, outside of the safe action area enclosed 
by border 40 6. Preferably, icon areas 4 04 are confined 
35 to corner regions each delimited by horizontal and 
vertical radial lines 410 originating at the center of 
curvature of the respective corners. These regions 
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provide adequate space for the icons while assuring that 
they will not be seen by home viewers. However, if 
desired, icon areas 4 04 can be located wholly or partly 
within border 406, but preferably outside of border 408. 
5 In this case., icons in areas 404 will create no more than 
minimal viewer distractions, particularly since, as will 
be described below, icons will normally lo^ present in a 
program element for only a period of several seconds. On 
the other hand, because icon areas 404 are located in the 

10 active picture window, they can be visually observed by 
personnel at a television station and, if played from tape 
in sufficiently slow motion, can be visually interpreted. 

One icon area 404a, for example at the upper left 
corner of area 402, may be reserved for a reference icon 

15 which may have a special shape and which is present in 
each television picture frame that contains identifying 
inf omnation. The information contained in the frame can 
be represented by the presence or absence of icons at the 
other three locations and/ or by the shape of the icon or 

2 0 icons at one or more of the other three locations. 

A simple example of an icon scheme according to the 
invention is shown in Fig, 4, where all of the icons have 
a square or rectangular outline. Such outlines can be 
produced in a relatively easy manner by gating circuitry 

25 as described above. The reference icon area 404 a is 
located at the upper left corner and has the form in the 
illustrated embodiment of a hollow black rectangle 
enclosing a hollow gray rectangle, with a white square at 
the center. The other three icons 404b-d are divided 

30 diagonally into a black triangle and a white triangle; to 
facilitate differentiation, the black triangles of icons 
in areas 404b and c point down and to the right; that of 
the icon in area 4 04d points up and to the left. 

Figs. 5-1-5.2 show a number of possible icon patterns 

3 5 which may be used in the practice of the present 

invention. Icon field patterns 5a-5h of Figs. 5. 1-5 • 2 
show the icon pixels present in Field 1 of a television 
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picture frame for eight unique icons, while icon field 
patterns 5i-5p show the corresponding icon pixels present 
in Field 2 of the same picture frame for the same eight 
icons. When displayed on a television screen, the lines 
5 of the associated icon field patterns are interlaced to 
depict a complete icon pattern. Thus, the field patterns 
5a and 5i are combined to form the reference icon pattern 
in area 404a of Fig. 4, while the icon field patterns 5b 
and 5j form the icon pattern in area 404d of Fig. 4 and 

10 icon field patterns 5c and 5k form the icon pattern in 
areas 404b and c of Fig* 4. Icon field patterns 5d and 
51, 5e and 5m, 5f and 5n, 5g and 5o, and 5h and 5p may be 
respectively combined to form five other icon patterns 
which may be employed. In the illustrated embodiment, 

15 white pixels are depicted without hatching, gray pixels 
with wide hatching and black pixels with close hatching. 

It is appreciated of course that other icon patterns 
may be employed. In the preferred embodiment, however, 

20 each icon comprises a plurality of pixels having different 
gray levels or other colors. Also, each icon pattern is 
relatively simple and has a relatively large number of 
pixels so as to provide a high degree of redundancy and 
imperviousness to signal degradation. 

25 Program information provided according to the present 

invention can be put to a variety of beneficial uses. By 
way of example, the information provided by icons 
according to the invention can be employed to assist 
analysis of edited program material. Because of the 

3 0 layering nature of video tape program editing, the source 
of the original material, i.e. the tape on which the 
original video information is stored, is sometimes 
difficult to determine. Re-editing of such a program 
would be facilitated by using icons to label every frame 

3 5 on a given reel of tape with a unique symbol or set of 
symbols. When the finished program is then played through 
an icon decoder, an accurate shot list can be assembled. 
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Machine readable commercial position information is 
also readily provided- During live broadcasts, such as 
sporting events, commercials are not transmitted at 
specific times, but float within the body of the program. 
5 Icons can be inserted at the originating point of the 
program to arm, or trigger, automatic commercial insertion 
equipment at every station which carries the program. In 
this manner, all local affiliates of a network or cable 
originator would remain in sync with the live programming. 

10 Icons can be employed to embed identification into 

programs and/or commercials. By embedding icons in 
finished programs or commercials, they can be monitored 
by off air receivers to make a log of actual vs, intended 
program material broadcast. In addition, bills to 

15 advertisers can be automatically generated. 

Selected icon patterns could be reserved to serve as 
smart test signals, or test signal identification patterns 
which would allow measurement equipment in television 
facilities to automatically make appropriate measurements 

20 on test signals as they pass through the measurement 
device. The information derived in this manner can be 
relayed to a node and then to the station control computer 
for use to control various processes in the station and 
to generate task lists and a database. 

2 5 During field production, icons can be employed as reel 

and take identifiers. The icon encoding circuitry 
described above can be adapted to operate in conjunction 
with portable video tape recorders either as stand alone 
equipment or as part of a video camera. By utilizing a 

3 0 portable data entry device such as a hand held computer, 

a director can enter reel, scene and take numbers into the 
video taping equipment just prior to shooting the scene. 
This information would go to an embedded icon encoder 
within the video recorder or camera so that all recorded 
3 5 frames would be labeled with the appropriate information. 

This information would be of great value in post 
production of the finished program and would constitute 
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a machine readable equivalen't of the clapboard and slate 
which are presently used- 

Icons can be employed to provide cross country routing 
information. Special instructions can be encoded in the 
5 icon format in accordance with the present invention, in 
the leader information of a program or commercial. This 
information would then be usable in real time by automatic 
switching equipment located in different parts of the 
country to route the signal to its destination without it 

10 being necessary for personnel at the various switching 
points to have prior knowledge of the specific route. 

While the description above refers to particular 
embodiments of the present invention, it will be 
understood that many modifications may be made without 

15 departing from the spirit thereof* The accompanying 
claims are intended to cover such modifications as would 
fall within the true scope and spirit of the present 
invention. 

The presently disclosed embodiments are therefore to 
20 be considered in all respects as illustrative and not 
restrictive, the scope of the invention being indicated 
by the appended claims, rather than the foregoing 
description, and all changes which come within the meaning 
and range of equivalency of the claims are therefore 
25 intended to be embraced therein. 
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WHAT IS CLAIMED IS : 

1- Apparatus for tracking the flow of television 
programs within a program distribution system, each 
progreucn including a video signal which contains 
information , to generate a television picture, the 
5 distribution system including a plurality of units through 
which the television programs flow, said apparatus 
comprising: 

first signal processing means connected within the 
system for adding signal elements to the video signal of 

10 a program in order to add patterns of nonbinary graphic 
symbols to the television picture such that the nonbinary 
graphic symbols are disposed at predetermined locations 
of the television picture and a plurality of successive 
patterns constitute a code uniquely identifying the 

1 5 program ; and 

second signal processing means connected within the 
system for detecting signal elements previously added to 
a video signal and for producing, in response to such 
detection, an identification of the program which included 

20 the video signal. 

2. Apparatus as defined in claim 1 further comprising 
control means, connected to said first and second signal 
processing means, for supplying to said first signal 
processing means information identifying a program which 

25 is flowing through said first signal processing means in 
order to control generation of associated signal elements 
by said first signal processing means. 

3. Apparatus as defined in claim 2 wherein said 
control means are connected for receiving program 

30 identifications from said second signal processing means. 

4. Apparatus as defined in claim 1 wherein the 
predetermined locations of the television picture are at 
the corners of the picture. 
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5. Apparatus as defined in claim 4 wherein one of the 
predetermined locations is a reference location for 
containing signal elements indicating the presence of 
program identifying information in a given picture fraiae. 
5 6- Apparatus as defined in claim 1 wherein each 

pattern of symbols is constituted by signal elements in 
two successive fields forming a television picture frame. 

7. Apparatus as defined in claim 1 wherein each 
pattern of symbols is part of an analog video signal, and 
10 said second signal processing means comprise analog- 
digital conversion means for converting signal elements 
previously added to the video signal into digital pixel 
signals, and detecting means for detecting the digital 
pixel signal. 

15 8. Apparatus as defined in claim 1 wherein the video 

signal is composed of a succession of picture fields 
separated by vertical blanking intervals and said second 
signal processing means perform detection of signal 
elements during vertical blanking intervals* 



20 



> 
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9- A method for tracking the flow of television 
programs within a program distribution system, each 
program including a video signal which contains 
information to generate a television picture, the 
5 distribution system including a plurality of units through 
which the television programs flow, said method 
comprising: 

adding, in first signal processing means connected 
within the system, signal elements to the video signal of 

10 a program in order to add patterns of nonbinary graphic 
symbols to the television picture such that the nonbinary 
graphic symbols are disposed at predetermined locations 
of the television picture and a plurality of successive 
patterns constitute a code uniquely identifying the 

15 program; and 

detecting, in second signal processing means connected 
within the system, signal elements previously added to a 
video signal and producing, in response to such detection, 
an identification of the program which included the video 

20 signal. 

10. An apparatus for use in connection with a video 
signal having signal elements representing icons, 
comprising: 

means for reading the icons of a video signal; and 
25 means for generating a report based upon the read 

icons . 

11, The apparatus of claim 10 wherein the video 
signal contains information to generate a video program 
and the icons represented by the signal elements of the 

3 0 video signal form a pattern which constitutes a code 
uniquely identifying the program and the report generated 
includes an identification of the program- 
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12. The apparatus of claim 10 wherein the video 
signal contains information to generate a plurality of 
video shots, each video shot having associated therewith 
video signal elements wherein the icons represented by the 

5 signal elements of the associated video shot form a 
pattern which constitutes a code uniquely identifying the 
shot and the report generated includes a shot list 
identifying each video shot of the of the video signal* 

13 . The apparatus of claim 10 wherein each icon is a 
10 picture of a graphic symbol comprising a plurality of 

pixels of different colors. 

14. A control apparatus for use in connection with 
one or more slave units and a video signal having signal 
elements representing icons, comprising: 

15 means for reading the icons of a video signal; and 

means for controlling a slave unit in response to the 
icons read from the video signal, 

15. The apparatus of claim 14 wherein the slave unit 
is a video tape recorder wherein the controlling means has 

20 means for activating the video tape recorder to record the 
video signal in response to icons read from the video 
signal . 

16- The apparatus of claim 14 wherein the slave unit 
is a router wherein the controlling means has means for 

2 5 activating the router to rout the video signal in 

accordance with the icons read from the video signal. 

17 . The apparatus of claim 14 wherein the slave unit 
is automatic commercial insertion equipment wherein the 
controlling means has means for activating the equipment 

3 0 to insert a commercial into the video signal in response 

to the icons read from the video signal - 

18. The apparatus of claim 14 wherein each icon is a 
picture of a graphic symbol comprising a plurality of 
pixels of different colors - 

3 5 19. The apparatus of claim 18 wherein at least one 

icon is a reference icon which provides reference levels 
for different colors of the icons. 
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20. The apparatus of claim 18 wherein the icon colors 
include white, gray and black, 

21. The apparatus of claim 19 wherein the means for 
reading the icons includes means responsive to the 

5 reference levels for comparing the icons of the video 
signal to predetermined icon patterns. 

22- Apparatus for use with a video signal which 
contains information to generate a television picture 
comprising a plurality of pixels, said apparatus 
10 comprising: 

an encoder for adding signal elements to the video 
signal in order to add graphic symbols to the television 
picture such that the symbols each comprise a plurality 
of pixels of a plurality of colors arranged in a 
15 predetermined pattern, and are disposed at predetermined 
locations of the television picture and a plurality of 
successive symbols constitute a coded message; and 

a decoder for detecting signal elements previously 
added to a video signal and for producing, in response to 
20 such detection^ a message decoded from the video signal. 

23* The apparatus of claim 22 wherein each symbol 
pattern includes pixels of different colors in at least 
two orthogonal directions- 
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